Digital cameras may capture color information using a color filter array (CFA) placed on top of an image sensor. A common CFA type is a Bayer array, which is based on a repetitive 2×2 pattern comprising one red (R), two green (G), and one blue (B) filters, with R and B color filters on one diagonal and two G color filters on the other diagonal. Recently introduced RGB-IR (red, green, blue-infrared) hybrid sensors may enhance this concept by capturing simultaneously both color information in the visible spectrum using traditional R, G, and B color filters and infrared (IR) information using IR filters, thus reducing the cost of the imaging system and the area required for multiple sensors. Generating a full color RGB image from RGB-IR data may involve a data filtering, interpolation, demosaicking, or restoration process. It is also possible to first generate the Bayer CFA image from RGB-IR data and then apply some Bayer demosaicking method to produce a full-color RGB image. Generating an IR image may involve extracting the IR pixels from RGB-IR data.
Technical challenges related to processing RGB-IR data may include the flexibility to operate with an arbitrary RGB-IR CFA layout, such as 2×2 and 4×4 repetitive CFA patterns and various other CFA layouts that may exist in future hybrid RGB-IR sensors. Other challenges may be related to image quality and computational efficiency. With respect to image quality, the results produced may include various color (for RGB processing) and edge artifacts (for both RGB and IR processing), whose presence typically depends on the scene complexity and processing performance, for instance.